The first people who banded into communities formed some overt and covert covenant with their neighbors, replete with trappings, that later became known as 'legal', in order to ensure their survival within this band of fellow strangers. The racial contract, though real, did not displace the social contract. Although the social contract, as per the French revolutionists terms of liberty, equality and fraternity never existed - even amongst Whites, it was always a myth and more symptomatic of Utopia - practical forms of the social contract, albeit with variations amongst gender and race, persisted. Integrated with the social contract, however, was a racial form of contract where male supremacism characterized by Christians of the white race gained sufficient power to dominate others who did not belong to their gender or race. Racial category, therefore (although not exclusive to Blacks or women) was, and is, a subcategory of the social contract. Mill reduces racial oppression to the single factor of Whites forming agreements amongst themselves to dominate the blacks, but, in reality, racial formation is a far more complex process with historical factors that included an African nation being colonized, exported as slaves, savagely treated (as were many others, and by other nations), exploited, and racially demeaned. The white man may have made himself superior, but his making himself so was not necessarily via contract, but, rather, by the realization that he was profiting form a certain system and loath to relinquish that system. Profiting from this system, too, gave him control over another and, natural to human nature, spurred his self-aggrandizement...

People in positions of power, often bully those they control.
Mill also states unilaterally that white supremacy was and is the political power of social control. Whilst this may have been true at one time when colonialism existed, White control has been thoroughly vetoed, and exists only arguably, and, then, in an implicit sense. Whilst America places much emphasis on race relations, the rest of the West seems to see no difference between whites and blacks and, possibly, due to their considering discussion of comparison trite and overdone, white supremacy hardly enters as topic in other countries (in fact, Islamaphobia may be more current at the moment). The U.S.A. might -- although this is arguable -- still contain traces of white superiority, but in most postcolonial states little if any distinction is made between blacks and that of any other race.

Mill's race contract has become an important work with wide popular appeal. There are many who point to it as political missive for their agenda. It seems to me, however, that whilst indicating content in some areas, the book generalizes and is simplistic and ill grounded in its basic theme. Firstly, race always existed but it may not necessarily have been due to contract. Secondly, social contract is a historical fact with rules varying in their application according to the dominant race and gender. Race contract was a Venn circle within the larger Venn diagram of social contract. The one contained the other.

Reference

Mills, C. (1997). The Racial Contract, Cornell University Press, Ithaca & London